Hierarchical Gaussian Process Regression
نویسندگان
چکیده
We address an approximation method for Gaussian process (GP) regression, where we approximate covariance by a block matrix such that diagonal blocks are calculated exactly while off-diagonal blocks are approximated. Partitioning input data points, we present a two-layer hierarchical model for GP regression, where prototypes of clusters in the upper layer are involved for coarse modeling by a GP and data points in each cluster in the lower layer are involved for fine modeling by an individual GP whose prior mean is given by the corresponding prototype and covariance is parameterized by data points in the partition. In this hierarchical model, integrating out latent variables in the upper layer leads to a block covariance matrix, where diagonal blocks contain similarities between data points in the same partition and off-diagonal blocks consist of approximate similarities calculated using prototypes. This particular structure of the covariance matrix divides the full GP into a pieces of manageable sub-problems whose complexity scales with the number of data points in a partition. In addition, our hierarchical GP regression (HGPR) is also useful for cases where partitions of data reveal different characteristics. Experiments on several benchmark datasets confirm the useful behavior of our method.
منابع مشابه
An Adaptive Hierarchical Method Based on Wavelet and Adaptive Filtering for MRI Denoising
MRI is one of the most powerful techniques to study the internal structure of the body. MRI image quality is affected by various noises. Noises in MRI are usually thermal and mainly due to the motion of charged particles in the coil. Noise in MRI images also cause a limitation in the study of visual images as well as computer analysis of the images. In this paper, first, it is proved that proba...
متن کاملPrediction with Gaussian Processes from Linear Regression to Linear Prediction and Beyond
The main aim of this paper is to provide a tutorial on regression with Gaussian processes We start from Bayesian linear regression and show how by a change of viewpoint one can see this method as a Gaussian process predictor based on priors over functions rather than on priors over parameters This leads in to a more general discussion of Gaussian processes in section Section deals with further ...
متن کاملPrediction with Gaussian Processes : from Linear Regression to Linear Prediction
The main aim of this paper is to provide a tutorial on regression with Gaussian processes. We start from Bayesian linear regression, and show how by a change of viewpoint one can see this method as a Gaussian process predictor based on priors over functions, rather than on priors over parameters. This leads in to a more general discussion of Gaussian processes in section 4. Section 5 deals with...
متن کاملHierarchical Bayesian Modelling with Gaussian Processes
We present a novel method for Gaussian process regression in a hierarchical Bayesian framework. The method is suited for cases where data from separate, but related problem scenarios is available. In the hierarchical framework, kernel matrices on a fixed set of input points (transduction) can be learned from data using a simple and efficient EM algorithm. This does not require a parametric form...
متن کاملHierarchical Mixture-of-Experts Model for Large-Scale Gaussian Process Regression
We propose a practical and scalable Gaussian process model for large-scale nonlinear probabilistic regression. Our mixture-of-experts model is conceptually simple and hierarchically recombines computations for an overall approximation of a full Gaussian process. Closed-form and distributed computations allow for efficient and massive parallelisation while keeping the memory consumption small. G...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2010